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PREFACE 


This  report  provides  results  for  a  field  study  conducted  at  Barksdale  AFB,  LA. 
The  study  showed  file  importance  of  enhanced  presentation  of  cannon  plug  data.  The 
effort  was  accomplished  as  a  cooperative  research  project  between  the  Air  Force  Institute 
of  Technology  (AFIT)  and  the  Air  Force  Research  Laboratory,  Logistics  Readiness 
Branch  (AFRL/HESR).  While  attaining  his  master’s  thesis,  Capt.  Bob  Webb  provided 
background  literature  reviews,  data  collection  and  analysis  of  the  data  obtained  for  this 
study.  Concept  development,  technical  direction,  hardware  and  software  development, 
and  Technical  Report  writing  were  accomplished  by  the  University  of  Dayton  Research 
Institute  (UDRI)  (Contract  No.  SP0900-94-D-0001,  DO.  0005)  and  AFRL/HESR. 

Special  thanks  to  the  2nd  Bomb  Wing  at  Barksdale  AFB,  LA  for  their  support  in 
this  effort. 
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REDUCING  CANNON  PLUG  CONNECTOR  PIN  SELECTION 
TIME  AND  ERRORS  THROUGH  ENHANCED  DATA 
PRESENTATION  METHODS 

SUMMARY 

This  research  investigated  the  effects  of  data  presentation  formats  on  maintenance 
technician  performance  when  maintenance  procedures  are  presented  on  a  monocular, 
head-mounted  display  (HMD).  Two  formats  were  used  to  present  the  maintenance 
procedure  information  to  the  technicians:  a  format  that  mimicked  the  standard  technical 
procedure  manual  format,  including  the  textual  and  graphical  aspects  of  the  format;  and  a 
format  that  provided  the  same  information  as  the  first,  while  adding  visual  cues  as  part  of 
the  graphical  portion  of  the  technical  information. 

United  States  Air  Force  (USAF)  avionics  maintenance  technicians  stationed  at 
Barksdale  Air  Force  Base,  Louisiana  served  as  subjects.  Dependent  measures  were  task 
completion  time,  task  error  rate,  and  technician  self  reports  on  the  usability  of  the 
presentation  methods  and  the  HMD.  Results  indicate  that  technicians  perform  tasks  more 
quickly  and  commit  fewer  errors  when  using  enhanced  graphical  data  presentation 
methods  to  perform  the  tasks.  The  technicians  indicated  that  HMDs  could  be  a  useful  tool 
in  the  performance  of  their  maintenance  duties. 


1 


INTRODUCTION 


Background 

Since  the  early  1980s  the  Department  of  Defense  (DoD)  and  commercial  industry 
have  been  moving  to  digitize,  computerize,  and  automate  the  technical  manuals  used  by 
maintenance  technicians  to  troubleshoot,  repair,  and  maintain  aircraft  systems.  Computer- 
based  maintenance  aids  that  provide  interactive  system  tests  and  access  to  technical 
information  currently  exist  in  the  operational  environment.  Ultimately,  such  systems  may 
become  the  primary  information  source  for  tasks  that  presently  require  the  use  of  paper- 
based  technical  documentation.  As  these  digitization  efforts  proceed,  the  logistics 
research  community  is  investigating  methods  to  efficiently  accomplish  this  task  while 
discovering  means  of  decreasing  logistics  costs. 

Military  Maintenance  Applications 

In  the  military  environment,  maintenance  is  continually  performed  in  less  than  ideal 
conditions.  For  example,  technicians  often  work  outdoors  in  a  wide  range  of  temperatures 
and  weather  conditions.  Furthermore,  specific  maintenance  tasks  may  require  completion 
within  a  confined  area,  such  as  a  landing  gear  or  engine  bay.  In  order  for  any  maintenance 
aid  to  be  a  true  asset,  it  must  be  usable  in  such  conditions.  Consequently,  most 
maintenance  aids  are  being  designed  as  ruggedized,  portable  laptop  computers  capable  of 
withstanding  extreme  environmental  conditions.  Improvements  in  hardware  and  software 
technologies  have  increased  the  viability  of  implementing  alternatives  configurations,  such 
as  wearable  computers  with  head-mounted  displays  (HMDs).  These  wearable  computers 
with  HMDs  have  been  shown  to  provide  the  mobility  necessary  to  perform  inspection 
tasks  more  accurately  than  a  portable  computer  (Friend  &  Grinstead,  1992).  Overall, 
research  has  shown  that  maintenance  aids  can  improve  technician  performance  in  problem 
identification,  task  completion  times,  and  reduced  maintenance  costs  (Schroeder,  Smith, 
Bursch,  &  Meisner,  1992;  Friend  &  Grinstead,  1992;  Thomas,  1995). 

Logistics  researchers  Wright-Patterson  AFB,  Ohio,  designed  and  specified  the 
technologies  necessary  for  an  Integrated  Maintenance  Information  System  (IMIS).  The 
IMIS  proof-of-concept  field  test  indicated  that  technicians  commit  a  significantly  higher 
number  of  errors  when  using  paper-based  maintenance  procedures  than  when  using 
electronically  presented  procedures.  However,  in  both  paper  and  electronic  conditions, 
correct  identification  and  selection  of  cannon  plug  pins  was  identified  as  a  highly  error- 
prone  task.  One  of  the  most  common  error  types  in  both  the  paper  and  electronic 
conditions  involved  reading  the  wrong  pin  on  a  cannon  plug.  Therefore,  although 
digitizing  technical  manual  does  improve  performance  overall,  there  are  still  areas  (such  as 
reading  cannon  plug  pins)  that  require  further  improvement. 

Several  automated  maintenance  aids  currently  exist  in  the  military  maintenance 
community.  Maintenance  Error  Decision  Aids  (MED A),  Avionics  Troubleshooting 
Systems  (ATS),  Flight  Control  Maintenance  Diagnostic  Systems  (FCMDS),  Aviation 
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Diagnostics  and  Maintenance  (ADAM)  systems,  and  Integrated  Maintenance  Information 
Systems  (IMIS)  are  all  examples  of  the  computer-based  aircraft  maintenance  aids  that  are 
beginning  to  appear  on  Air  Force  flightlines  (Hibit  and  Marx,  1994;  Gulick  and  Kell,  1993; 
Schroeder,  Smith,  Bursch,  and  Meisner,  1992;  Le  Beau,  et.  al.,  1991;  Thomas,  1995). 

HMD  Research  in  the  Aircraft  Maintenance  Environment 

In  1991,  Masquelier  compared  the  use  of  an  HMD  to  a  flat-screen  computer 
during  routine,  intermediate-level  bench  top  troubleshooting  tasks.  In  this  stationary 
position,  no  statistical  differences  were  found  in  performance  between  the  group  that  used 
the  HMD  and  the  group  that  used  the  flat-screen  display.  However,  Masquelier  noted  that 
a  similar  study  with  mobility  requirements  might  result  in  performance  differences. 

Friend  and  Grinstead  (1992)  conducted  the  follow-up  study  recommended  by 
Masquelier.  Working  in  a  flightline  maintenance  environment,  technicians  were  required 
to  move  around  the  airplane  to  conduct  inspection  and  fault  isolation  tasks.  As  with 
Masquelier,  aircraft  maintenance  technical  data  was  presented  either  on  an  HMD  or  flat- 
screen  display.  The  important  difference  between  Friend’s  and  Grinstead’ s  (1992)  and 
Masquelier’s  (1991)  studies  is  that  technicians  in  Friend’s  and  Grinstead’s  (1992)  study 
conducted  tasks  on  an  actual  flightline,  in  normal  flightline  working  conditions.  In  the 
flightline  environment,  which  requires  increased  mobility,  technicians  using  the  HMD 
completed  the  task  more  quickly  and  accurately  than  the  technicians  that  used  the  flat- 
screen  display.  Cannon  plug  pin  tests  are  commonly  performed  in  mobile  flightline 
maintenance;  therefore,  due  to  Friend’s  and  Grinstead’s  finding,  this  study  used  a  HMD  as 
the  display  device  of  choice. 

HMD  Advantages  and  Disadvantages 

When  compared  to  traditional  flat-screen  displays,  HMDs  offer  several  advantages, 
ultimately  leading  to  increased  user  task  effectiveness  as  reflected  by  reduced  errors  and 
reduced  task  completion  times  (Friend  and  Grinstead,  1992).  The  primary  disadvantages 
associated  with  HMD  usage  involve  physiological  and  perceptual  side  effects  that  often 
accompany  prolonged  (e.g.,  several  hours)  HMD  use. 

HMD  Advantages.  The  advantages  of  HMDs  are  compactness,  low  weight,  low 
cost,  and  rugged  characteristics.  When  used  with  a  portable,  wearable  computer  system, 
the  HMD  enables  increased  user  mobility  and  improve  task  performance  (Friend  and 
Grinstead,  1992).  The  user  can  thus  directly  interact  with  the  environment  while 
simultaneously  operating  the  computer  via  the  HMD  (Quill,  Kancler,  &  Masquelier, 

1995).  In  the  aircraft  maintenance  environment,  such  mobility  provides  advantages  over 
portable,  laptop  computer  formats,  especially  during  tasks  which  have  a  hands-free 
requirement. 

Improved  technician  performance  leads  to  several  benefits  (Ebling,  1997; 

Langford,  1995).  For  example,  reduced  maintenance  time  increases  equipment 
availability.  Improvements  in  task  completion  times  may  also  enable  a  reduction  in  total 
manpower  requirements  without  over-stressing  an  already  limited  maintenance  manpower 
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pool.  In  general,  if  the  cost  to  maintain  a  system  can  be  reduced,  possible  future  budget 
cuts  can  be  absorbed  without  sacrificing  maintenance  effectiveness.  As  shown  by 
Raaijmakers  and  Verduyn  (1996),  technicians  with  relatively  little  training  can  be  as 
effective  as  experienced  technicians  when  presented  with  maintenance  aids  that  enable 
system  understanding  and  comprehensive  data  presentation  for  task  performance.  This 
may  ultimately  reduce  training  requirements  and  thus  reduce  the  overall  cost  needed  to 
prepare  maintenance  technicians. 

HMD  Disadvantages.  The  side  effects  experienced  from  prolonged  HMD  use 
include  retinal  rivalry,  reduced  visual  field,  difficulty  with  depth  perception,  and  increased 
eye  stress  and  fatigue  (Rash  and  Martin,  1988;  Hale  and  Piccione,  1990;  Haworth  and 
Newman,  1993;  Kotulak  and  Morse,  1995).. 

Retinal  rivalry  involves  the  reduced  ability  to  selectively  switch  back  and  forth 
between  separate  images  being  presented  to  each  eye  (Rash  and  Martin,  1988).  This  is  an 
important  factor  related  to  the  use  of  a  monocular  HMD  where  the  user  must  switch 
between  the  HMD  display  in  one  eye  and  the  working  environment  in  the  other.  Use  of 
HMDs  for  periods  amounting  to  a  normal  maintenance  shift  may  result  in  adverse  effects. 

Visual  resolution,  depth  perception,  and  field  of  view  are  all  impacted  by  placing 
an  HMD  directly  in  front  of  one  eye  and  nothing  in  front  of  the  other.  Adjustable  HMDs 
can  be  physically  aligned  to  allow  the  user  to  see  around  it  (Haworth  and  Newman,  1993; 
Kotulak  and  Morse,  1995).  But  while  adjusting  the  HMD  may  improve  some  effects,  it 
often  causes  others.  Eye  strain  and  fatigue  often  accompany  long-term  use  of  HMDs 
(Hale  and  Piccione,  1990).  These  factors  are  also  often  compounded  by  the  effect  on  the 
wearer’s  depth  perception  associated  with  HMD  use  (Hale  and  Piccione,  1990). 

In  dynamic  environments,  such  as  the  flightline,  HMD  wearers  must  not  only 
adjust  to  having  a  display  screen  directly  in  front  of  one  eye,  but  must  also  continue  to 
monitor  their  changing  surroundings.  Attention  must  be  divided  or  shifted  between 
external  stimuli  and  the  data  presented  on  the  HMD  screen.  As  the  wearer  becomes  tired, 
the  eyes  are  less  able  to  quickly  adjust,  and  performance  (and  safety)  may  be  compromised 
(Kotulak  and  Morse,  1995). 

General  Dynamics  (1991),  Masquelier  (1991),  and  Friend  and  Grinstead  (1992) 
indicated  that  users  did  experience  a  slight  degree  of  eye  strain;  however,  this  was 
attributed  to  the  subjects’  unfamiliarity  with  the  wear  and  use  of  the  HMD.  Participants 
also  indicated  that  they  had  some  interference  between  the  HMD  display  and  objects  in  the 
environment  on  which  they  were  trying  to  focus.  While  test  participants  indicated  that 
they  initially  had  such  problems,  they  also  indicated  that  after  becoming  accustomed  to 
wearing  the  HMD,  they  were  able  to  eliminate  these  problems. 

None  of  the  studies  indicated  the  percentage  of  participants  that  experienced  these 
difficulties.  The  coverage  of  these  issues  in  each  report  indicated  that  the  actual  problems 
were  minor  and  applied  to  few  test  subjects.  However,  any  indication  of  such  problems 
should  be  of  potential  concern,  thus  the  current  research  effort  also  queried  the  test 
subjects  regarding  visual  problems  during  or  after  participation. 
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Maintenance  Technicians 

In  their  studies,  both  Masquelier  (1991)  and  Fiend  and  Grinstead  (1992)  measured 
experience  level  as  a  between  subject’s  variable.  In  both  studies,  technicians  were 
identified  as  having  more  experience  or  less  experience  as  a  maintainer.  Both  studies 
found  that  experienced  aircraft  maintenance  technicians  statistically  took  less  time  and 
made  fewer  mistakes  than  those  technicians  with  limited  experience.  Thus,  the  current 
study  was  limited  to  experienced  technicians  because  it  was  felt  that  experienced 
technicians  are  familiar  with  the  task  and  can  thus  focus  on  a  more  in-depth  analysis  of  the 
presentation  method  and  HMD.  Inexperienced  technicians  would  likely  need  to  divide 
attention  between  the  potentially  unfamiliar  maintenance  task  and  the  conditions  of  the 
study,  ultimately  curtailing  their  ability  to  provide  quality  feedback. 

Technical  Data  Presentation 

A  primary  objective  of  this  research  was  to  determine  if  maintenance  technicians’ 
cannon  plug  pin  selection  and  connection  performance  can  be  improved  through  the  use  of 
an  enhanced  technical  data  presentation  method. 

Display  presentation  properties  for  graphical  and  scientific  data  have  been 
extensively  studied.  Factors  such  as  display  background,  contrast  levels,  viewing  distance, 
size,  orientation,  and  amount  of  irrelevant  information  on  the  display  have  long  been 
identified  as  areas  of  concern.  When  a  technician  is  required  to  make  decisions  based  on 
graphically  displayed  data,  these  items  must  be  carefully  monitored  and  controlled. 

In  the  current  study,  textual  data  was  presented  in  both  upper  and  lower  case 
letters  (Helander,  1988).  Studies  by  Bennett  and  Flach  (1992),  Boles  and  Wickens 
(1987),  and  Payne  and  Lang  (1995)  have  shown  that  mixed-format  displays  (graphics  and 
text  combinations)  produce  better  performance  results  than  either  text  or  graphics  alone; 
therefore,  for  the  current  study,  all  conditions  presented  both  graphics  and  text. 

Wickens,  Merwin,  and  Lin  (1994)  found  that  combining  2-D  graphics  with  visual 
cues  (color,  bold  lettering,  highlighting,  shading)  provides  the  best  performance  results. 
Raaijmakers  and  Verduyn  (1996)  found  that  the  performance  of  Naval  engineers  was 
improved  during  unfamiliar  fault  detection  and  identification  with  a  help  system  that 
combined  graphics  with  visual  cues.  Tufte  (1997)  discusses  the  advantages  of  visually 
“deactivating”  the  background  and  minimizing  its  visual  characteristics  in  relation  to  items 
in  the  foreground.  The  current  study  used  a  combination  of  these  visual  enhancements, 
such  as  highlighting  and  shading  as  visual  cues.  As  part  of  the  highlighting  scheme, 
background  information  (cannon  plug  pins  not  to  be  tested)  were  grayed  out,  along  with 
the  image  of  the  cannon  plug  housing. 

Galitz  (1985)  states  that,  “Specific  areas  of  the  screen  should  be  reserved  for 
certain  kinds  of  information,  such  as  commands,  error  messages,  and  input  fields,  and 
maintain  these  areas  consistently  on  all  screens.”  Galitz  also  recommends  that  for 
programs  that  display  multiple  screens  with  similar  information,  the  information  should  be 
consistently  grouped  by  on-screen  location.  Furthermore,  while  the  current  study  focused 
on  providing  enhancements  to  existing  digitized  paper-based  information,  only  the 
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information  essential  to  the  situation  was  included  (Gordon,  1968;  Helander,  1988;  Tufte, 
1997;  Yonas  and  Pittenger,  1973). 


The  Present  Study 

The  goal  of  the  present  study  was  to  evaluate  technician  performance  while  using 
an  HMD  on  the  highly  error-prone  continuity  check  task  involving  cannon  plug  pin 
identification. 

The  present  study  used  a  Kopin™  HMD,  a  monochrome  display.  The  data 
presented  was  limited  to  only  the  exact  information  needed  to  perform  the  tasks  requested 
in  the  test  program.  The  Kopin™  HMD  also  had  an  adjustable  eyepiece  that  allowed  the 
wearer  to  set  optimal  viewing  distance,  thereby  eliminating  visual  acuity  problems 
associated  with  viewing  displays  at  improper  distances  (Giese,  1946;  Quill,  Kancler,  & 
Batchelor,  1996). 

Experienced  maintenance  technicians  were  exclusively  used  for  the  current  study. 

Two  electronic  display  conditions  were  created:  a  one-to-one  representation  of 
the  images  as  seen  on  the  current  paper-based  technical  manuals,  and  a  display  format 
using  visual  cues  and  enhancements.  Because  cannon  plugs  vary  greatly  in  size  and 
number  of  pins,  a  second  variable  was  added  to  represent  “many-pin”  and  “few-pin” 
cannon  plugs. 

Performance  was  measured  in  terms  of  task  completion  time  and  correct  cannon 
plug  identification.  A  post-test  questionnaire  was  also  administered  to  collect  qualitative 
information  on  the  technicians’  use  of  the  HMD  and  software  systems. 
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METHOD 


Subjects 

Avionics  maintenance  technicians  from  the  2nd  Bomb  Wing  at  Barksdale  AFB,  LA 
served  as  test  subjects.  The  technicians  were  volunteers  chosen  from  a  pool  of  available 
personnel.  Criteria  for  consideration  were  based  on  time  in  service,  time  on  the 
flightline/bench,  current  duties,  and  eyesight.  Technicians  needed  a  minimum  of  four  years 
continuous  active  duty  experience,  must  have  been  working  on  the  flightline/bench  for  at 
least  two  years,  and  have  been  currently  performing  maintenance  duties  on  the 
flightline/bench.  Technician  eyesight,  corrected  to  20/20,  was  used  as  a  minimum 
baseline.  While  a  vision  exam  was  not  administered,  technicians  were  verbally  queried  as 
to  whether  they  had  corrected  20/20  vision. 

Ocular  dominance  (dominant  sighting  eye)  of  the  technician  was  not  tested,  nor 
was  the  HMD  required  to  be  used  on  the  dominant  eye.  Experiments  indicate  that  while 
performing  static  maintenance  tasks,  technician  performance  is  not  affected  by  which  eye 
(dominant  versus  non-dominant)  views  the  HMD  (Kancler  &  Quill,  1997).  However, 
because  the  findings  do  suggest  that  switching  the  HMD  from  eye  to  eye  does  affect 
performance,  technicians  in  the  current  study  were  discouraged  from  switching  the  HMD 
from  eye  to  eye  in  the  middle  of  a  task. 

Hardware  and  Software 

Several  hardware  components  were  used  in  this  research.  The  display  device  was 
a  commercially  available  monocular  display  manufactured  by  Kopin™.  A  486-based 
laptop  computer  served  as  the  CPU.  The  test  apparatus  comprised  a  total  of  five  multi-pin 
cannon  plug/breakout  box  assemblies.  A  single,  rectangular  assembly  was  used  for 
training.  The  remaining  four  circular  assemblies  were  used  during  the  experiment. 

The  software  used  in  the  experiment  was  a  locally-developed  test  program  coded 
in  Visual  Basic™  4.0.  A  menu-driven  test  program  provided  the  text-based  procedures, 
the  accompanying  graphical  images,  and  (when  appropriate)  visual  enhancements  to  the 
graphical  images. 

Two  individuals  were  responsible  for  administering  the  experiment.  The  test 
administrator  controlled  the  experiment  software  via  a  laptop  computer.  The 
experimenter  was  responsible  for  correct  arrangement  of  the  cannon  plugs  prior  to  each 
trial,  instruction  to  the  subject,  and  inspection  for  subject  errors  at  the  end  of  each  trial. 

Apparatus 

Four  different  circular  cannon  plugs  were  used  in  the  testing  phase  of  this  research; 
each  plug  contained  12, 13,  55,  or  79  “female,”  (recessed)  pins.  Female  cannon  plugs 
were  exclusively  chosen  to  reduce  task  variance,  and  because  the  indexing  and  visual 
references  are  more  consistently  marked  and  readable.  The  four  cannon  plugs  used  in  the 
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testing  phase  were  divided  into  two  groups:  ‘few’  pins  (12  and  13)  and  ‘many’  pins  (55 
and  79).  Figure  1  depicts  the  front  view  of  the  ‘few’  pin  cannon  plug  layouts  and  Figure  2 
depicts  the  front  view  of  the  ‘many’  pin  cannon  plug  layouts. 


Figure  1:  Graphics  of  Cannon  Plug  Faces  with  ‘Few’  Pins 


Figure  2:  Graphics  of  Cannon  Plug  Faces  with  ‘Many*  Pins 

Each  group  contained  one  numerically  indexed  and  one  alphabetically  indexed 
cannon  plug.  While  the  indexing  scheme  of  the  cannon  plugs  was  not  tested,  the  various 
indexing  strategies  were  selected  to  be  a  representative  sample  of  what  technicians 
encounter  in  the  field. 

For  each  trial,  subjects  were  required  to  locate  and  identify  two  specific  pins  on 
each  cannon  plug.  Four  different  pin  combinations  for  each  cannon  plug  were  used 
throughout  the  test.  A  random  number  generator  was  used  to  select  the  first  pin 
combination  for  each  cannon  plug.  The  remaining  three  combinations  were  selected  based 
on  inter-pin  distance  and  relative  location  on  the  cannon  plug.  This  pin  selection  scheme 
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was  intended  to  create  a  set  of  pin  combinations  that  represented  varied  distances  and 
orientations. 

Two  technical  data  presentation  methods  were  employed:  enhanced  and 
unenhanced.  Both  presentations  contained  the  same  basic  information  that  is  presented  in 
typical  paper  technical  manuals:  a  picture  of  the  cannon  plug  in  question,  index 
references,  and  a  table  of  the  appropriate  pins  to  connect.  The  enhanced  presentation 
contained  additional  visual  cues  that  highlighted  or  emphasized  the  pins  to  connect  while 
de-emphasizing  non-relevant  data.  Figure  3  contains  examples  of  both  presentation 
methods  for  the  55  pin  cannon  plug. 


Figure  3:  Enhanced  and  Unenhanced  HMD  Presentation  Methods 


Experimental  Design 

The  research  methodology  was  a  2  x  2  within-subjects  randomized  block  design. 
The  independent  variables  were  Pin  Number  (few  [12,  13  pins]  vs.  many  [55,  79  pins]) 
and  Presentation  type  (enhanced  vs.  unenhanced).  Pin  combination  (each  pair  of  pins  to 
be  tested  on  the  cannon  plug)  was  used  as  a  control  variable  and  not  included  in  the 
statistical  analysis.  Subjects  received  all  possible  combinations  of  the  four  cannon  plug 
sizes  and  two  presentation  types,  resulting  in  a  total  of  sixteen  trials  per  subject. 

Order  of  presentation  was  obtained  through  a  randomized  block  methodology. 

This  method  was  selected  because  the  subject  pool  was  not  large  enough  to  accommodate 
a  complete  counterbalancing  of  all  possible  conditions  (Keppel  &  Zedeck,  1989).  The 
block  randomization  was  accomplished  as  follows: 

1)  The  16  trials  to  be  performed  by  each  technician  were  divided  into  groups  of 
eight,  containing  two  of  each  cannon  plug  size  (12, 13,  55,  and  79  pin)  and  one 
of  each  of  the  presentation  types  (enhanced  and  unenhanced)  for  each  cannon 
plug  pair. 

2)  Each  group  of  eight  trials  was  then  divided  in  half.  Each  of  the  resulting  four 
trials  contained  one  of  each  of  the  four  cannon  plug  sizes. 

3)  Trials  were  then  randomly  ordered  according  to  rules  one  and  two  above. 
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Dependent  variables  were  completion  time  and  errors.  Task  completion  time  was 
defined  as  the  time  elapsed  between  the  subject’s  verbal  indication  of  task  initiation  and 
task  completion.  Errors  were  defined  as  an  incorrect  identification  of  either  of  the  cannon 
plug  pins.  Additional  qualitative  information  was  also  collected  in  the  form  of  a  post-test 
questionnaire. 


Procedure 

Subjects  were  required  to  complete  a  personal  background  form.  Subjects  then 
received  a  briefing  on  the  experimental  procedure  and  training  on  the  use  of  the  HMD. 

The  technicians  completed  a  practice  session  using  a  rectangular  cannon  plug.  During  the 
practice  session,  technicians  were  exposed  to  the  exact  procedures  used  during  the 
experiment. 

Once  the  subject  completed  the  practice  session  and  indicated  an  appropriate  level 
of  familiarity  with  the  HMD,  the  experimental  session  was  initiated.  Although  no  time 
limit  was  placed  on  task  completion,  subjects  were  asked  to  work  as  quickly  and  as 
accurately  as  possible.  To  provide  a  common  starting  point,  subjects  were  allowed  to 
orient  the  cannon  plug  and  layout  the  appropriately  sized  test  adapters  before  each  trial 
began.  Subjects  were  also  reminded  that  the  experiment  was  concerned  with  task 
performance  and  not  the  operation  of  the  HMD.  If  a  subject  was  experiencing  difficulty  in 
operating  the  HMD  or  any  of  its  components,  the  experimenter  provided  assistance  prior 
to  the  beginning  of  the  trial.  If  the  HMD  or  any  of  its  related  components  failed  during 
the  trial,  or  the  subject  was  unable  to  complete  the  full  set  of  trials  for  any  reason,  that 
subject’s  data  was  eliminated  from  the  analysis. 

After  orienting  the  cannon  plug  and  locating  the  adapters,  the  subject  indicated  to 
the  program  administrator  to  start  the  timer.  As  per  the  presentation  on  the  HMD,  the 
subject  inserted  test  adapter  connectors  into  the  appropriate  cannon  plug  pins.  Upon 
completion,  the  subject  indicated  to  the  program  administrator  to  stop  the  timer.  The 
experimenter  then  used  a  multimeter  to  check  for  errors  in  pin  identification.  Throughout 
the  experiment,  the  experimenter  recorded  any  additional  observations  (technician 
comments,  adjustments  made  to  the  HMD)  on  an  experimenter  observation  form. 
Following  the  test,  subjects  completed  the  post-test  questionnaire. 

Hypotheses 

The  overall  research  hypothesis  was  that  graphically  enhanced  data  presentations 
would  improve  technician  cannon  plug  pin  selection  and  connection  task  performance. 

The  following  hypotheses  detail  how  technician  performance  was  defined: 

I.  Few-pin  cannon  plugs  will  result  in  faster  task  completion  times  than  many-pin 
cannon  plugs. 

II.  The  graphically  enhanced  presentation  on  the  HMD  will  result  in  faster  task 
completion  times  than  the  unenhanced  presentation. 

HI.  In  the  unenhanced  condition,  there  will  be  greater  task  completion  times  for 
many-pin  cannon  plugs  than  for  few-pin  cannon  plugs.  In  the  enhanced 
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presentation  condition,  task  completion  times  will  be  the  same  for  few-  and 
many-pin  cannon  plugs. 

IV.  Few-pin  cannon  plugs  will  result  in  less  errors  than  many-pin  cannon  plugs. 

V.  The  graphically  enhanced  presentation  on  the  HMD  will  result  in  fewer  errors 
than  the  unenhanced  presentation. 

VI.  In  the  unenhanced  condition,  there  will  be  a  greater  number  of  errors  for  many- 
pin  cannon  plugs  than  for  few-pin  cannon  plugs.  In  the  enhanced  presentations 
condition,  errors  will  be  the  same  for  few-  and  many-pin  cannon  plugs. 
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RESULTS 


Overview 

Twenty-eight  aircraft  avionics  maintenance  technicians  stationed  at  Barksdale 
AFB,  LA  performed  16  cannon  plug  pin  selection  and  connection  tasks  (i.e.,  16  trials 
each).  Quantitative  and  qualitative  data  were  collected  from  all  of  the  participating  test 
subjects.  The  results  of  the  quantitative  data  analysis  are  presented  and  discussed  in  this 
section.  The  qualitative  data  that  were  collected  for  all  of  the  28  participating  technicians 
through  the  user  evaluation  questionnaire  and  the  structured  interview  pertain  to  the 
usability  of  the  HMD  system  and  a  presentation  of  information  on  HMDs.  These  findings 
are  detailed  in  the  discussion  section. 

Four  of  the  technician’s  quantitative  data  sets  were  excluded  from  analysis.  One 
was  discarded  because  the  technician  was  given  the  wrong  cannon  plug  during  two  of  the 
tasks.  The  quantitative  data  was  determined  unusable  because  of  the  procedural  error.  A 
second  technician’s  data  was  eliminated  because  the  test  program  sequencing  was 
improperly  programmed  and  the  wrong  graphical  presentations  were  presented  to  the 
technician  during  the  test.  Although  the  technician  performed  the  tasks  properly,  the 
incorrect  presentation  resulted  in  the  experimental  design  being  compromised.  The  other 
two  data  sets  were  discarded  because  both  of  the  technicians  indicated  during  the 
structured  interview  that  they  did  not  see  the  enhancements  on  the  graphics. 

Consequently,  the  data  collected  for  these  subjects  was  incomplete  and  could  not  be  used. 

Dependent  Measures. 

Data  was  analyzed  using  a  repeated  measures  analysis  of  variance  (ANOVA) 
procedure  on  SPSS™  7.5  for  Windows  95™.  Independent  variables  were  number  of  pins 
(few  vs.  many)  and  presentation  type  (unenhanced  vs.  enhanced).  Dependent  measures 
(analyzed  separately)  were  total  task  time  and  errors. 

Task  Time.  Results  for  the  task  time  data  analysis  are  presented  in  Table  1 . 
Timing  began  through  a  verbal  cue  from  the  subject,  after  the  subject  had  selected  and 
oriented  the  appropriate  cannon  plug.  Timing  ended  when  the  subject  verbally  indicated 
task  completion. 


Table  1.  ANOVA  summary  table  for  task  completion  times. 


Source  of 

Variance 

df 

■ 

Critical 

F-Value 

Calculat 

ed 

F-Value 

P-Value 

Pin  Number 

8223.25 

i 

8223.25 

4.280 

44.79 

<.001 

Within 

4223.06 

23 

183.61 

■aaina 

34941.59 

1 

34941.59 

4.280 

92.44 

<.001 

|  Within 

8693.73 

23 

377.99 

4995.38 

1 

4995.38 

4.280 

39.62 

<.001 

!  Within 

2900.18 

23 

126.09 
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Cannon  Plug  Pin  Number.  The  main  effect  of  pin  number  was  significant 
[F(l,23)=44.79,  p<.001].  Few-pin  cannon  plugs  (those  with  12  and  13  pins)  yielded 
shorter  task  completion  times  than  larger  cannon  plugs  (those  with  55  and  79  pins). 

Presentation  Type.  The  main  effect  of  presentation  type  was  also  significant 
[F(l,23)=92.44,  p<.001].  As  hypothesized,  the  enhanced  display  format  yielded  shorter 
task  completion  times  than  the  unenhanced  display  format. 

Pin  Number  X  Presentation  Type.  As  hypothesized,  the  interaction  of  pin  number 
X  presentation  type  was  significant  [F(l,23)=39.62,  p<.001].  In  the  unenhanced 
presentation  condition,  many-pin  plugs  were  associated  with  significantly  greater  task 
completion  times  than  few-pin  plugs.  In  the  enhanced  presentation  condition,  no 
statistically  significant  difference  in  task  completion  times  was  observed  between  few-  and 
many-pin  cannon  plugs  (Figure  4). 


■Few 

-Many 


Figure  4.  The  interaction  of  pin  number  X  presentation  type  on 

total  task  time  in  seconds. 

Error  Rate.  Results  for  the  error  data  analysis  are  presented  in  Table  2.  Error  rate 
was  calculated  based  on  the  technician’s  ability  to  correctly  identify  the  proper  pins  the 
first  time.  Technicians  were  allowed  only  one  attempt  per  trial.  To  control  for  data 
collection  variance,  the  same  experimenter  conducted  all  the  data  collection  activities. 
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Table  2.  ANOVA  summary  table  for  errors. 


Source  of 

Variance 

df 

Mean 

Square 

Critical 

F-Value 

Calculat 

ed 

F-Value 

P-Value 

Pin  Number 

.128 

i 

.128 

4.280 

1.205 

.284 

Within 

2.435 

23 

.11 

3.19 

1 

3.19 

4.280 

20.255 

<.001 

j  Within 

3.62 

23 

.16 

■Bna 

2.19 

i 

2.19 

4.280 

23.73 

<.001 

|  Within 

2.12 

23 

.09 

Cannon  Plug  Pin  Number.  The  main  effect  of  pin  number  was  not  significant 
[F(l,23)=1.205,  p>.05].  Contrary  to  hypothesis,  few-pin  cannon  plugs  (those  with  12  and 
13  pins)  did  not  yield  fewer  errors  than  many-pin  cannon  plugs. 

Presentation  Type.  The  main  effect  of  presentation  type  was  significant 
[F(l,23)=20.255,  p<.001].  As  hypothesized,  the  enhanced  display  format  yielded  fewer 
errors  than  the  unenhanced  display  format. 

Pin  Number  X  Presentation  Type.  As  hypothesized,  the  interaction  of  plug  size  X 
presentation  type  was  significant  [F(l,23)=39.62,  p<.001].  In  the  unenhanced 
presentation  condition,  many-pin  plugs  were  associated  with  a  statistically  significant 
greater  error  rate  than  few-pin  plugs.  In  the  enhanced  presentation  condition,  no 
statistically  significant  difference  in  error  rate  was  observed  between  few-  and  many-pin 
cannon  plugs  (Figure  5). 
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Figure  5.  The  interaction  of  pin  number  X  presentation  type  on 

task  error  rate. 
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DISCUSSION 


Quantitative  Results 

Task  Time.  The  overall  mean  task  completion  time  data  collected  for  this  thesis 
supports  the  hypotheses  that  the  enhanced  data  presentation  enabled  technicians  to  better 
complete  the  experimental  task. 

Pin  Number,  For  task  time,  the  significant  main  effect  of  pin  number  was  likely 
due  to  several  factors,  the  most  notable  being  the  dramatic  difference  in  the  actual  number 
of  pins.  The  second  factor  is  the  combination  of  the  cannon  plug  pin  size  and  cannon  plug 
assembly  size.  For  few-pin  cannon  plugs,  the  average  plug  assembly  size  was  slightly 
smaller  than  the  many-pin  cannon  plugs;  however,  for  the  few-pin  cannon  plugs,  average 
pin  size  was  greater  than  the  many-pinned  cannon  plugs.  Also,  for  the  many-pin  cannon 
plugs,  pin  density  (number  of  pins  per  area)  was  certainly  higher.  In  terms  of  target 
detection,  for  the  few-pin  plugs  the  ratio  of  target  area  (size  of  specified  pins)  to  non¬ 
target  area  (the  remaining  pins  plus  the  area  of  the  cannon  plug  face)  was  1  to  20.  For  the 
larger  plugs,  this  ratio  was  1  to  60.  This  disparity  in  “target-to-noise”  ratio  may  have 
contributed  to  the  overall  effect  of  cannon  plug  size  on  task  completion  times. 

Presentation  Type.  For  task  time,  the  significant  main  effect  of  presentation  type  is 
consistent  with  the  notion  that  visual  cues  such  as  highlighting  will  assist  in  target 
detection.  Furthermore,  the  enhancement  scheme  called  for  the  de-highlighting  (graying 
out)  of  non-target  pins.  As  per  Wickens,  Merwin,  and  Lin  (1994),  Raaijmakers  and 
Verduyn  (1996),  and  Tufte  (1997),  the  combination  of  these  strategies  produce  the 
expected  result  of  reduced  target  detection  times. 

Pin  Number  X  Presentation  Type.  The  significant  interaction  of  pin  number  X 
presentation  type  indicates  that  the  many-pin  plugs  benefit  more  from  the  enhanced  display 
condition.  It  is  possible  that  the  smaller  plug  sizes  yielded  a  “floor  effect,”  while  the 
many-pin  plugs  approached  worst  case.  The  present  study  did  not  include  several  cannon 
plug  formats.  Future  research  may  address  cannon  plugs  with  intermediate  numbers  of 
pins  to  establish  a  potential  “critical  limit”  at  which  graphical  enhancements  significantly 
improve  performance. 

Error  Rate.  Similar  to  task  times,  overall  task  error  data  collected  for  this  thesis 
supports  the  hypotheses  that  the  enhanced  data  presentation  enabled  technicians  to  better 
complete  the  experimental  task. 

Pin  Number.  The  non-significant  finding  of  this  main  effect  may  have  been  due  to 
a  possible  speed/accuracy  tradeoff.  Subjects  were  told  to  work  as  quickly  and  accurately 
as  possible;  however,  quicker  completion  times  may  have  been  sacrificed  for  higher 
accuracy,  especially  in  the  many-pin  cannon  plug  conditions.  This  scenario,  when  coupled 
with  a  potential  few-pin  “floor  effect,”  may  have  led  to  this  non-significant  finding. 

Presentation  Type.  As  with  task  time,  the  significant  main  effect  of  presentation 
type  on  errors  was  consistent  with  the  findings  of  Wickens,  Merwin,  and  Lin  (1994)  and 
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Raaijmakers  and  Verduyn  (1996)  indicating  graphical  enhancements  yield  improved 
performance. 

Pin  Number  X  Presentation  Type.  The  significant  interaction  of  pin  number  X 
presentation  type  follows  a  pattern  similar  to  the  task  time  results:  the  many-pin  plug  sizes 
benefit  more  from  the  enhanced  display  condition.  Again,  it  is  possible  that  the  smaller 
plug  sizes  yielded  a  “floor  effect,”  while  the  larger  plug  sizes  approached  worst  case.  The 
conclusion  to  be  drawn  from  the  time  and  error  data  can  only  be  interpreted  to  the  extent 
that  enhancements  to  12-  and  13 -pin  connector  graphics  do  not  lead  to  improved 
performance,  while  enhancements  to  55-  and  79-pin  connector  graphics  do. 

Qualitative  Results 

Data  in  the  form  of  a  questionnaire  and  interview  were  collected  from  all  28  of  the 
test  participants.  Most  of  the  data  was  directed  at  the  usability  the  information  presented 
on  the  HMD  and  comfort  of  use  of  the  HMD. 

Information  Presented  on  the  HMD.  The  technicians  were  asked  if  the 
information  presented  on  the  HMD  was  readable  and  if  and  when  they  noticed  a  difference 
in  presentation  types  (unenhanced  vs.  enhanced).  All  28  technicians  indicated  that  the 
information  itself  was  mostly  legible.  Some  references,  such  as  numbers  and  the  extreme 
comers  of  the  HMD  were  said  to  be  “fuzzy;”  but,  all  indicated  that  the  HMD  was 
adequate  in  presenting  the  data. 

Some  technicians  indicated  apprehensiveness  about  relying  on  the  enhanced 
presentations.  The  reasons  given  were  lack  of  trust  in  the  presentation,  the  plugs  were 
easy  to  read,  and  they  felt  more  comfortable  (and  familiar  with)  counting  the  pins  out. 

The  technicians  indicated  that  the  pin  counting  and  mistrust  of  presentation  were  the  result 
of  encountering  numerous  technical  orders  that  were  improperly  marked  and  that  they  did 
not  inherently  trust  technical  data. 

HMD  Use.  The  technicians  in  this  study  only  wore  the  HMD  system  for 
approximately  30  minutes.  The  tasks  were  performed  in  a  static  environment  with  few 
outside  distractions  or  interference.  Even  so,  the  technicians  provided  insightful 
comments  about  the  HMD  and  its  potential  role  in  the  flightline  maintenance  environment. 

Technicians  were  asked  about  the  HMD  and  their  ability  to  use  it  with  their  current 
job  requirements.  The  primary  concerns  expressed  by  the  technicians  were  potential  eye 
strain  caused  by  switching  focus  between  the  cannon  plug  and  HMD.  Additionally, 
several  extra  clips  and  accessories  that  were  part  of  the  HMD  assembly  interfered  with  the 
HMD  use  (an  earpiece  and  microphone  were  not  used).  Finally,  the  weight  of  the  system 
was  seen  as  a  potential  cause  of  headaches  and  sweating,  resulting  in  the  HMD  slipping 
down  on  the  forehead. 

Even  with  these  concerns,  the  technicians  overwhelmingly  indicated  that  the  HMD 
would  be  an  asset  to  them  in  performing  their  daily  duties.  Twenty-three  of  the 
technicians  said  the  HMD  could  be  used  in  troubleshooting,  schematic  tracing,  or  tasks 
that  required  their  hands  to  be  free.  Other  suggested  uses  were  in  dark,  tight  places. 
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during  night-time  operations,  and  on  tasks  that  required  two  people  to  perform.  The 
suggestion  was  that  technicians  could  be  electronically  connected  to  the  same  reference 
and  could  work  together  better  by  using  the  HMD. 

Technicians  felt  that  having  wiring  diagrams,  schematics,  and  checklists  on  an 
HMD  would  enable  technicians  to  concentrate  on  the  task  and  not  the  data  needed  to 
perform  the  task.  As  one  technician  put  it,  some  tasks  require  one  hand  to  hold  a 
flashlight,  one  hand  to  hold  the  test  leads,  one  hand  to  hold  the  screwdriver,  one  hand  to 
operate  the  test  equipment,  and  one  hand  to  turn  the  page  in  the  TO  --  “even  an  octopus 
would  be  hard  pressed”  to  do  the  task  properly. 

Other  suggestions  included  hooking  the  HMD  up  to  an  on-line  system  that  would 
enable  the  wearer  to  access  any  and  all  the  information  needed  for  the  job.  Suggested 
information  included  supply  information,  core  automated  maintenance  system  data,  all  the 
aircraft  maintenance  history,  the  World  Wide  Web,  and  e-mail.  Incorporating  the  HMD 
into  a  complete  system  of  miner’s  light,  ear  defenders,  radio,  and,  inter-phone  was  also 
frequently  mentioned. 
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CONCLUSIONS  AND  RECOMMENDATIONS 

The  results  of  this  research  suggest  that  enhanced  cannon  plug  pin  selection  task 
data  presented  on  an  HMD  would  be  useful  in  improving  maintenance  technician 
performance.  Technicians  completed  the  cannon  plug  pin  selection  and  connection  tasks 
quicker  and  committed  fewer  errors  when  they  used  the  enhanced  data  presentation  to 
perform  the  task  while  wearing  the  HMD.  Analysis  of  the  obtained  data  revealed 
statistically  significant  results  that  indicated  that  the  enhanced  data  presentation  method 
did  reduce  the  task  performance  time  and  error  rate.  Such  performance  improvements 
could  ultimately  lead  to  reduced  maintenance  costs.  Improved  technician  performance  can 
lead  to  benefits  such  as  increases  equipment  availability,  reduction  in  total  manpower 
requirements,  and  reduced  costs  associated  with  logistics  requirements. 

Qualitative  data  obtained  from  technician  feedback  also  suggest  that  HMD  systems 
would  be  advantageous  for  and  used  by  technicians  in  the  performance  of  their 
maintenance  duties.  Technicians  reported  that  they  encountered  very  few  problems  using 
the  HMD  and  would  like  to  see  some  type  of  HMD  system  available  for  their  use. 

Future  Research 

Future  research  should  incorporate  a  wider  variety  of  cannon  plug  formats, 
including  higher  varieties  of  pin  numbers,  plug  shapes,  and  indexing  schemes.  For 
example,  the  cannon  plugs  in  the  present  study  were  selected  to  represent  the  extreme 
cases  of  pin  numbers  (few  and  many).  Results  from  the  current  study  showed  differences 
in  technician  performance  between  many-  and  few-pin  cannon  plugs  for  unenhanced 
presentations,  while  there  were  no  differences  for  the  enhanced  presentations.  Future 
research  could  set  up  intermediate  pin  sizes  and  establish  at  what  pin  number  graphical 
enhancements  significantly  improve  performance.  Additional  graphical  enhancement 
strategies  could  also  be  investigated,  such  as  color  coding  or  flashing  indicators. 

The  portability  of  the  current  HMD  system  was  not  tested  to  its  fullest  extent  in 
the  present  study.  Subjects  did  not  have  to  move  around  to  perform  the  required 
maintenance  tasks.  Future  research  should  investigate  flightline  maintenance  tasks  and 
other  tasks  requiring  increased  mobility  using  enhanced  presentations.  As  possible  future 
avenues  of  study,  the  technicians  in  the  present  study  suggested  system  troubleshooting 
tasks,  inspection  tasks,  schematic  and  troubleshooting  chart  reading  tasks,  and  tasks  that 
required  coordinated  teams  of  technicians  operating  concurrently. 

The  results  of  the  present  study  indicated  that  improved  task  information 
presentation  methods  can  improve  technician  performance,  ultimately  resulting  in  reduced 
maintenance  costs  and  increased  equipment  availability.  While  the  current  study 
uncovered  methodologies  that  will  reduce  errors,  these  methodologies  did  not  completely 
eliminate  errors.  Furthermore,  a  cost-effective  strategy  must  be  developed  to  integrate 
these  methodologies  into  present  and  future  digitization  efforts.  For  these  reasons,  further 
research  in  the  presentation  of  task  information  and  cannon  plug  design  should  be 
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pursued,  with  the  ultimate  goals  of  task  time  improvement,  error  reduction,  and, 
ultimately,  maintenance  cost  reduction. 
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